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parameter  estimator  figures  significantly  in  CIE  coverage  performance  -  the  less  bias  the 
better.  A  secondary  role  is  played  by  the  marginal  distribution  of  the  stationary  process.  We 
also  point  out  that  not  all  CIE's  are  equal  -  some  require  fewer  observations  before 
manifesting  the  properties  for  CIE  validity. 


Subject  classifications:  Simulation,  statistical  analysis,  statistical  estimation,  time  series, 
small  sample  behavior  of  confidence  intervals 


DUDLEY  KNOX  LIBRARY 
NAVAL  POSTGRADUATE  SCHOOL 
_]_  MONTEREY,  CALIFORNIA     93943-5002 


This  paper  studies  the  small  sample  behavior  and  convergence  properties  of 
a  number  of  confidence  interval  estimators  (CIE's)  for  the  mean  \i.   of  a 
stationary   process,  X, , ,X  .  These  CIE's  are  typically  of  the  form 

Pr^  e  \   *-   td,l-«/2(?/n)*i  *  '  "  «'  (1) 

where  X  =  E.X./n,  t  ,  „  is  the  Y-quantile  of  the  t-distribution  with  d  decrees 

n         1   r  d,Y  *  b 

of  freedom  (d.o.f.),    and  V  estimates  a    =  nVar(X  )   (or  the  variance  parameter 

\  /  n      v  n  r 

2  2  2      2 

a  =  lim    a  ) .  A  "good"  estimator  for  a  (or  a  )  is  the  cornerstone  of  a 
n-«o  n      *  n 

valid  CIE  for  fi.  Many  estimators  have  been  studied  in  the  context  of  discrete- 
event  simulation:  nonover lapping  batch  means  (NOBM)  (Conway  1963,  Schmeiser 
1982,  Kang  1984);  independent  replications;  overlapping  batch  means  (OBM) 
(Meketon  1980,  Meketon  and  Schmeiser  1984);  standardized  time  series  (STS) 
(Schruben  1983,  Goldsman  1984,  Glynn  and  Iglehart  1990);  spectral  theory 
(Fishman  1973,1978,  Heidelberger  and  Welch  1981,1983);  ARMA  time  series 
modeling  (Fishman  1973,1978,  Schriber  and  Andrews  1984);  and  regeneration 
(Crane  and  Iglehart  1975,  Crane  and  Lemoine  1977,  Fishman  1978). 

There  is  considerable  work  which  compares  the  various  CIE  methodologies. 
The  Monte  Carlo  (MC)  work  mainly  deals  with  small  sample  CIE  performance;  see, 
e.g.,  Law  and  Kelton  (1984)  and  Goldsman,  Kang,  and  Sargent  (1986).  Analytical 
results  are  almost  all  asymptotic:  Goldsman  and  Schruben  (1984),  Goldsman  and 
Meketon  (1986),  Damerdji  (1987),  Glynn  and  Iglehart  (1990),  and  Schmeiser  and 
Song  (1989)  all  compare  various  combinations  of  the  CIE's. 

In  this  paper  we  study  finite  sample  behavior  of  CIE's  from  NOBM,  OBM,  and 
STS.  Section  1  gives  background  material.  Section  2  reports  on  statistical 
properties  of  various  variance  estimators.  Section  3  presents  analytical 
results  for  some  special  cases  and  then  summarizes  a  MC  study  of  the  CIE's.   In 
Part  I  of  our  MC  work,  we  break  the  n  observations  into  b  batches,  and  we 
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observe  what  happens  as  the  batch  size  grows.  For  "small"  b,  NOBM  performs  the 
best  with  respect  to  CIE  coverage.     For  "large"  b,  both  NOBM  and  OBM  fare  the 
best.  However,  an  STS  method  produces  CIE's  with  smaller  expected  lengths. 
Another  comparison  is  carried  out  in  Part  II  of  our  MC  work,  where  we  fix  the 
d.o.f.  d  and  observe  what  happens  as  n  grows;  here,  the  NOBM,  OBM,  and  STS 
"combined"  CIE's  perform  similarly.  Section  4  discusses  our  findings,  and 
Section  5  summarizes.  We  conclude  that  the  bias  of  V  is  the  most  important 
factor  in  determining  a  CIE's  validity;  a  secondary  role  is  played  by  the 
marginal  distribution  of  the  X.'s.  We  also  find  that  a  CIE  having  superior 
large  sample  properties  may  have  relatively  poor  small  sample  performance.  We 
offer  practical  and  research  recommendations. 

1.  BACKGROUND 

We  review  the  CIE's  and  stochastic  processes  under  study.  We  assume  the 

stochastic  processes  satisfy  certain  moment  and  mixing  conditions,  as  described 

2    2 
in  the  cited  papers.  We  henceforth  use  the  notation  Nor(u,T  ),  X  (d),  X(d), 

Exp(X),  and  t(d)  to  represent  the  normal,  chi-square,  chi,  exponential,  and  t 

distributions,  each  with  the  appropriate  parameters. 

1.1  NonoverlappinE  Batch  Means 

Suppose  we  divide  X1 , — ,X  into  b  >  1  adjacent,  nonover lapping  batches  of 

size  m  (assume  n  =  bm) .  The  i-th  batch  mean,    i = 1, — ,b,  is  X.   = 
v  '  i,m 

7.  .   X,.  „>.   ./m.   In  implementing  the  method  of  NOBM,  we  assume  the  X.   's  are 
L)=l     (i-l)m+j'         r       •  1>m 

2  2 

approximately  i.i.d.  Nor(ji,a  /m) .  The  NOBM  estimator  for  a  is 

VN  =m  lh.  =  1[   X.>m  -  Xj2/(b-l)  5  a2X2(b-l)/(b-l), 
where  "  -*  "   denotes  convergence  in  distribution  as  m  ■+   ®.  The  NOBM  CIE  for  \i 
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is  given  by   (1)   with  d  =  b-1   and  V  =  V 


r 

1.2  Overlapping  Batch  Means 

Define  the  i-th  overlapping  batch  mean,    i =  1, . . . ,n-m+l,  by  X(i,m)  = 

ym~J  X.  ./m.   The  OBM  estimator  for  a2  is 
^=0   i+j' 

VQ  =  nm  J*Im+1  (X(i,m)  -  lj2/[(n-m+l)(n-i»)]  , 

and  is  almost  identical  to  Bartlett's  spectral  estimator  (see  Priestley  1982). 

The  OBM  CIE  for  \i   is  given  by  (1)  with  V  =  V_;  its  validity  depends  on  V_  being 

2  2 
approximately  a  X  (d)/d.  Meketon  and  Schmeiser  (1984)  take  d  =  1.5*(b-l),  where 

b  =  n/m.  Based  on  MC  experimentation,  Schmeiser  (1986)  recommends  d  = 

1.5*(b—  l)[l+(b-l)     *   ];  we  shall  use  this  value  in  our  MC  work. 

1.3  Standardized  Time  Series 

Suppose  X. ,  —  ,X  is  divided  into  b  ;>  1  adjacent,  nonover lapping  batches 
of  size  m.   For  i  =  l,  —  ,b,  let  A.  =  Jfv,  [(m+l)/2  -  jJXx.^  ...   Schruben 

(1983)  assumes  the  A.'s  are  approximately  i.i.d.  normal,  and  proposes  the  area 

2 
and  combined   estimators  for  a  : 

V  =   «    yb   A2  2  °ZxZ(b)     and  V  =  ^lll^A     $  ^(Zb-l) 
VA  "  ,  3  v.  *-i=l   i       b     ana  VC  "     2b-l  2b-l 

(m  -m)b 

CIE's  for  \l   are  formed  by  substituting  the  appropriate  V  and  d  in  (1).  Schruben 

2 
also  derives  the  so-called  "maximum"  estimator  for  a  (see  Subsection  4.2). 

1.4  Some  Time  Series  Processes  of  Interest 

We  will  have  occasion  to  use  the  following  ARMA-type  processes. 

MA(1):  X.  =  £.  +  9£.  „,   where  c.  ~  i.i.d.  Nor(0,l)  and  -1  <  6  <  1. 
1    l     l-l  l  \  »  / 

MA'(l):  X.  =  c.  +  ©c.^,  where  c.  ~  i.i.d.  Exp(l)  and  -1  <  9  <  1. 
AR(1):  Xi  =  cpXi:l  +  c.,  where  c.  ~  i.i.d.  Nor(0,l-(p2) ,  -K(p<l. 
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<pX.«       w.p.  <p       where  e.  ~  i.i.d.  Exp(l)  and 
EAR(l):  X.  =  j  »  X 

(pX.   +  e.   w.p.  l-(p      0  <,   (p  <  1   (see  Lewis  1980). 

We  also  consider  other  stationary  normal  processes  as  well  as  the  waiting  time 
(delay)  processes  of  M/M/l  and  E  /M/l  queues. 

1.5  CIE  Performance  Criteria 

Denote  the  NOBM,   OBM,    area,    and  combined  CIE's  by  CIEM,   CIE~,   CIE.,    and 

W      U      A 

CIEp,  resp.   The  half-length   of  a  generic  CIE  is  H  =  t .  ,_  ,2(V/n)  *.  We  use 
the  following  CIE  performance  measures:  coverage  (Pr^  E  I  +  Hf),  E[H],  and 
Var(H).   Among  CIE's  which  achieve  coverage  1  -  a,  we  prefer  that  with  the 
smallest  E[H],  and  then  that  with  the  smallest  Var(H). 


2.  PROPERTIES  OF  VARIOUS  VARIANCE  ESTIMATORS 

We  give  some  results  on  the  bias  and  variance  of  the  NOBM,  OBM,  and  STS 
estimators,  and  on  the  asymptotic  performance  of  the  corresponding  CIE's. 

2.1  Bias  of  the  Estimators 

2        ~     2     ~ 
The  bias  of  V  as  an  estimator  for  a  is  Bias(V)  =  a  -  E[V].   Goldsman  and 

Meketon  (1986)  show  that  lim    m lim,   Bias(V„)  =  lim    m  lim,   Bias(V_)  = 
x    '  m-*»     b-Ko     v  N       m-Ko     b-»a>      0 

lim    m  lim,   Bias(V_)/2  =  lim    mBias(V.)/3.  All  of  these  estimators  are 
asymptotically  unbiased  as  m  ■+  <»,  but  for  small  m,  these  estimators  can  be 
quite  biased. 

2 
Example  1:  For  the  AR(1)  and  EAR(l),  the  Appendix  gives  a  =  (l+<p)/(l-<p)  and 

E[5]  =  a2  -  SSiSliL  +  m£±5ty  -,  c2  -  -*£-,   for  large  .  and  b,    (2) 
"        mb(l-cf>r    m(b-l)(l-»)'i        m(l-ip) 
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m(b-l)(l-1p)3 


mb  -i        .    /A     mw,     mb-m+1 


^  +  is.    +  *«K*-<p  Ilia L  (3) 

b^     -I       m(b-l)  (mb-m+1)  (l-<p) 


=  a * — _  for   large  m  and  b. 

m(l-(p) 


24<p f-m  +l-<p  (m+1)       <p-(m+l-m<p)<p        1    .     2         6<p  ^ 

(m3-m)(l-(p)2  L  4  (l-<p)2  J  m(l-<p)2 


x„m+l 

E[VA]  =  </  + 


E[Vf]  =  a2  +  . 
C  (2b 


for   large  m, 
mb     «i_ /■    ,  «\„IB     « rt,    r  <    /    .  <        \   m- 


2<p  [    lb  ,  lV"     2b(m+2)<pffl  ,  12b<p[l-(m+l-mcp)<pffl;M  (5) 

-l)m(l-(p)2L  b  V-1  (m2-l)(l-(p)2        J 


=  a * — s  for   large  m  and  b. 

m(l-(p) 

If  m  and  b   are   large,   the  bias  results  anticipated  by  Goldsman  and  Meketon  are 

attained.     Table   1  contains  exact  E[V]'s  for  b  =  2  and  16  and  various  m.     For 

small  b,   Bias(VN)   <  Bias(V0)   <  Bias(Vc)   <  Bias(V);   for   large  b,   Bias(v\.)   = 

Bias(V-)   <  Bias(Vr)   <  Bias(V.).     The  biases  become  negligible  as  m  grows.    ,, 

2       2 

Example  2:  For  the  MA(l)  and  MA'(l),  the  Appendix  gives  a     =   (1+0)   and 

E[VN]  =  a2  -  29(b+l)/mb  =  a2  -  29/m  for  large  b.  (6) 

E^  =  *2  "  irf^TJ  Vir  ~  mb^m+l]  *  °2 "  2e/m    for  lar*e  b-  w 

E[VA]  =  ct2  -  69/m.  (8) 

E[VC]  =  a2  -  (4b+2)0/mb  =  a2  -  40/m  for  large  b.  (9) 

The  conclusions  from  Example  1  again  hold.  ,  , 

Although  Bias(V)  is  interesting  in  its  own  right,  the  bias  directly 
affects  CIE  performance.  Consider  the  unrealistic  case  in  which  m  is  fixed  and 
b  -►  co.  For  the  estimators  studied  here,  one  can  show  that  as  b  -»  <o,  V  -►  E[V] 
w.p.l,  and  so  T  =  (1   -|i)/(V/n)*  2  Nor(0,a2/E[V]) .  Falsely  assuming  T  ~ 
Nor(0,l)  as  b  ->  co  yields  incorrect  CIE  coverage, 
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2vX 


Prill  EMHi,  2*[ZH/2(E[V]/(J)]  "  1  ■  (l0) 

where  *(•)  and  zy  are  the  Nor(0,l)  c.d.f.  and  Y-quantile.   If  E[V]/a2  <  [>]  l, 

then  coverage  <  [>]  1  -  a.   (Coverage  is  quite  sensitive  to  decreases  in 

*   2 
E[V]/a  .)  So  the  less  bias  the  better.  As  b  -»  »,  CIEM  and  CIE~  tend  to 

w       u 

achieve  the  desired  coverage  more  quickly  with  respect  to  m  than  do  CIE.  and 
CIEC;  see  Sargent,  Kang,  and  Goldsman  (1987)  (S-K-G) . 

2.2  Variance  of  the  Estimators 

4 
Goldsman  and  Meketon  report  that  as  m  and  b  become  large,  b*Var(V„)  -►  2a  , 

b-Var(V0)  -►  4a4/3,  b-Var(VA>  ->  2a4,  and  b-Var(Vc)  -►  a4  (cf.  Damerdji  1987). 

For  i.i.d.  X1,...,Xn>  Kang  and  Goldsman  (1990)  find  Var(VN)  and  Var(VA) 

exactly,  as  do  Song  and  Schmeiser  (1989)  for  Var(V_).   Exact  results  for  other 

processes  and  for  Var(Vfl)  are  tedious  to  derive.   Of  course,  one  can  also 

calculate  the  mean  squared  errors  of  the  V's  (cf.  Schmeiser  and  Song  1989). 

2.3  Asymptotic  Properties  of  the  CIE's 

Schmeiser  (1982)  and  Goldsman  and  Schruben  (1984)  note  that  as  m  ->  ®, 

(mb)%H  2  <rtd>1_a/2X(d)Ald  (11) 

for  CIEN,  CIE.,  and  CIEp.   An  analogous  approximate   result  holds  for  CIEn- 
Under  (11),  the  CIE's  achieve  coverage  1  -  a  as  m  ■*  ».   Further,  if  r(«)  is  the 
gamma  function,  then 

0»b)*E[H]  +  otdl.a/2(2/d)X  r%)lf)   ,   and  (13) 

->V«(H)  *  chl  W2  {  .  -  f  [  Oj$HP  f  }  .  m 

The  right  sides  of  (12)  and  (13)  decrease  in  d   and,  hence,  in  b.   So  for  large 
m  with  fixed  b,  E[HN]  >  E[HA]  >  E[HQ]  >  E[HC]  and  Var(h*N)  >  Var(HA)  >  Var(HQ)  > 
Var(Hp),  the  subscripts  havine  the  obvious  meanings.   Goldsman  and  Schruben 
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(1984)  and  Meketon  and  Schmeiser  (1984)  let  b  -»  «  in  (12)  and  (13)  to  get 

1,     H  =  h\ 


E[H]  Var(H) 


lim^  =   1,   H  =  HQ,  HA,  Hc,      and     lim^ =  i  2/3,   H  =  Hn   .  (14) 


m-K» 


r     ,        *•  "0'"A'"C>  ^"varOn  ° 

ELHNJ  dh»  Var^V        [l/2,   H  =  HC 


A 


Thus,  as  b  also  becomes  large,  all  of  the  CIE's  have  about  the  same  E[H]'s,  but 
CIEf  has  the  smallest  Var(H). 


3.   FINITE  SAMPLE  CONFIDENCE  INTERVAL  ESTIMATION 

Small  sample  analysis  of  CIE's  is  difficult.  We  present  a  few  exact 
results,  but  most  of  the  section  is  devoted  to  a  MC  study. 

3.1  Some  Analytical  Results 

Example  3:  Suppose  Xj,...,!  ~  i.i.d.  Nor(fi,T2).   Then  VN  ~  T2X2(b-l)/(b-l) , 

y\  ~  T2X2(b)/b,  and  V„  -  T2X2(2b-l)/(2b-l) .  Further,  I     is  independent  of  VM, 
A  L  n  IN 

V.,  and  Vn  (see  Appendix).   So  (1)  holds  exactly  for  CIEM,  CIE.,  and  CIEn.   We 

A         L  H  A  L 

could  not  obtain  such  results  for  CIE0  or  for  nonnormal  i.i.d.  processes.  ,, 

Example  4:  We  can  derive  exact  results  for  CIE.  when  b  =  1  (n  =  m)  and  X,,,...,X 

is  stationary  normal.     Then  A,  ~  Nor(0,E[A2]) ,  V.  ~  E[V.]X  (1),  and  1     is 

l  l     A     A  n 

normal  and  independent  of  V.  (see  Appendix).  So  (X  -ii)(nc/V.)  *  ~  t(l),  where 

A  II  A 

c  =  E[V.]/cr  .      Hence,   the  coverage  of  CIE.    is  2Pr$t(l)£t4    4      /0>Jc£-l  = 

An  A  1,1~ Of/*; 

(2/n)Tan  (t.  «   /?^c)*   (As  *n  (*0)»  the  coverage  is  sensitive  to  decreases 
in  c.)   Similarly,  E[H.]  =  t.  «   /2(2E[V.]/nn)  .   To  illustrate,  suppose  the 
X.'s  are  AR(l).   Figure  1(a)  uses  Example  1  and  (A-3)  to  plot  coverage  vs. 
log2n  for  1  -  a  =  0.90  and  various  <p.  We  see  that  for  <p  >  [<]  0,  the  coverage 
is  <  [>]  1  -  a.  As  |<p|  approaches  0  or  as  n  grows,  Bias(V.)  decreases,  and  the 
coverage  approaches  1  -  a.  Figure  1(b)  has  analogous  plots  of  E[H.]  vs. 
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2  -x 

log2n.      If  <p  =  0.0,   then  E[V  ]  =  a    =  1,   and  so  E[H.]   decreases  at  rate  n     . 

*  2  -X 

If  <p  =  -0.9,   then  E[V  ]   decreases  to  a    =  1/19,   and  E[H.]   decreases  at  rate  n 

(after   initially  decreasing  faster).     The  ip  =  0.9  plot   for  E[H.]   increases  and 

*  2 

tijen  decreases  as  n  grows.     This  occurs  since  E[V.]    increases  to  a     =19  as 

n  ■♦  »,   while  the  -Jn   in  E[H.]rs  denominator  becomes   large.     // 

Example  5:   We  give  exact  results  for  CIEN  when  b  =  2  (n  =  2m)   and  X1 , ,X     is 

stationary  normal.     Then  VM  =  mS.(X.      -t  )2/(b-l)  =  m(X\      -X0     )2/2;   so  VM  ~ 

N     ii, mn  l  ,m   c  ,m         n 

"■   2  = 

E[VN]X  (1).   Since  X   is  normal  and  independent  of  V,.  for  b  =  2  (see  Appendix), 

we  have  (X  -u)(nc '/V«)  *  ~  t(l),  where  c'  =  E[VN]/o  .   The  coverage  is 

(2A)Tan"1(t1  ^^c*),  and  E[HN]  =  ^  ^gCE^/Hm)*.   K  the  X.'s  are 

2 
AR(1),  then  (A-3)  yields  a  ,  and  (2)  with  b  =  2  gives  E[V„];  c',  coverage,  and 

E[HN]  then  follow.   These  performance  measures  behave  as  in  Example  4.  ,  , 

It  is  difficult  to  generalize  the  above  CIE  results  to  b  >  2,  since  we 

2 
would  then  have  correlated   X  random  variables.   Thus,  we  only  gave  exact 

results  for  simple  cases.  We  resort  to  MC  experimentation  in  the  sequel. 

3.2  Desi£n  of  the  Monte  Carlo  Study 

Our  goal  was  to  assess  CIE  performance  over  a  variety  of  stochastic 
processes  and  choices  of  number  of  observations  n,  batch  size  m,  d.o.f.  d,  and 
desired  coverage  1  -  a.   We  simulated  the  following  processes:  AR(l)  with 
<p  =  0.0, +0.1, +0.5, ±0.9;  EAR(l)  with  (p  =  0.0,0.1,0.5,0.9;  MA(1)  and  MA'(l)  with 
e  =  +0.1, +0.9;  11/11/ 1  with  traffic  intensity  p  =  0.6,0.8  (service  rate  =  1.0); 
and  E„/M/l  with  p  =  0.6  (service  rate  =  1.0).  Each  run  was  initialized  from  the 
appropriate  steady  state   distribution.  All  uniform  [normal]  variates  were 
generated  from  algorithm  UNIF  [TRPNRM]  in  Bratley,  Fox,  and  Schrage  (1987); 
exponentials  used  inversion. 
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For  ease  of  exposition,  we  divided  the  study  into  two  parts.   In  Part  I  we 
fixed  1  -  a  and  b,  and  then  charted  CIE  coverage  as  a  function  of  m.  Roughly 
speaking,  we  wanted  to  know  which  CIE  first  achieved  acceptable  coverage  as  id 
increased  with  fixed  b.  Further,  which  CIE's  had  the  best  E[H]  and  (to  a 
lesser  extent)  Var(H)?  For  the  stochastic  processes  discussed  above,  we 
conducted  at  least  1000  independent  runs  of  16384  observations;  these  runs  were 
used  to  calculate  CIEM,  CIE,.,  CIE.,  and  CIE,-,  and  the  resulting  performance 

N      U      A  L 

characteristics  for  all  choices  of  m  =  2  ,  k  =  0,1, — ,10,  b  =  1,2,4,8,16,  and 
1  -  a  =  0.80,0.90,0.95,0.99. 

In  Part  II,  we  set  1  -  a  =  0.90  and  d  =  3  and  15,  and  then  charted 
coverage  as  a  function  of  n.  For  d  =  3  [d  =  15],  CIEN  uses  b  =  4  [b  =  16], 
CIE,.  uses  b  =  2  [b  =  11],  CIE.  uses  b  =  3  [b  =  15],  and  CIEn  uses  b  =  2 

U  A  L 

[b  =  8].   We  conducted  our  experiments  on  a  number  of  stochastic  processes, 
each  of  which  used  2000  runs  of  16384  observations  to  calculate  the  CIE's  for 
all  n  =  2k,  k  =  4,..., 14,  and  d  =  3  and  15.   Since  d  is  fixed,  (ll)-(13) 
suggest  that  all  of  the  CIE's  will  perform  about  the  same  for  large  m;  however, 
we  suspected  that  they  would  exhibit  different  small  sample  performance  because 
the  variance  estimators  incorporated  in  the  CIE's  operate  under  different 
assumptions.   We  first  discuss  the  underlying  assumptions  for  Vv,  V.,  and  V_ 

W    A         L 

since  these  estimators  require  independence  between  batches  (\L  does  not).   The 

estimator  V.,  [V.   assumes  that  the  X.   's  [A.'s]  are  i.i.d.  normal;  the 
N  L  AJ  i,m   L  l  J 

combined  estimator  V_  must  satisfy  both  assumptions.   For  fixed  d  and  n,  V„  and 

V.  use  roughly  half  the  batch  size  of  V-;  hence,  the  assumption  of  i.i.d. 

X.   's  [A.'s]  is  harder  to  achieve  for  V„  [V.'l  than  for  V_.  On  the  other  hand, 
i,mi  IN   A  C 

V  's  assumption  of  normality  of  the  X.   's  is  easier  to  satisfy  than  V  's 

N  '         i,m  '       A 

assumption  of  normality  of  the  A.'s  which  relies  on  a  more  restrictive 
functional  central  limit  theorem.   The  normality  question  for  V„  vs.  V-  is  not 
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as  clear  since  V,.  uses  half  of  V  's  batch  size.   CIEn  appeals  to  spectral 

2  2 
theory  to  directly  assume  that  V«  is  o  X  (d)/d;  this  supposition  is  not  true 

for  nonnormal  processes  or  for  finite  batch  sizes. 

3.3  Results  from  Part  I  of  the  Monte  Carlo  Study 
3.3.1  Representative  results 

We  discuss  typical   results  from  Part  I.  Figures  2,  3,  and  4  illustrate 
CIE  performance  when  X1>...,X  are  AR(l)  (<p  =  0.9),  EAR(l)  (<p  =  0.9),  and  M/M/l 
(p  =  0.8),  resp.,  and  1  -  a  =  0.90.   The  AR(l)  and  EAR(l)  have  the  same 
covariance  function,  but  the  AR(1)  has  normal  marginals  while  the  EAR(l)'s  are 
exponential.   The  M/M/l 's  joint  distribution  is  more  complicated.  We  only 
consider  the  cases  b  =  2  and  16  since  CIEN,  CIE~,  and  CIEp  require  b  ;>  2,  and 
since  one  can  argue  that  b  =  16  is  "large"  (at  least  for  the  "usual"  choices 
of  a).   Each  of  Figures  2,  3,  and  4  has  four  graphs:  (a),(b)  are  for  b  =  2,  and 
(c),(d)  are  for  b  =  16.   In  (a)  and  (c),  we  plot  the  sample  coverage  (CVG)  vs. 
log2m;  (b)  and  (d)  plot  the  sample  E[H]  (EHL)  vs.  log2m.   The  standard  error  of 
any  CVG  is  about  [CVG-Cl-CVO/lOOO]*. 

Figure  2  is  for  the  AR(l)  with  <p  =  0.9.  All  CVG's  are  poor  for  small  m, 
but  approach  1  -  a  =  0.90  as  m  increases.  For  b  =  2,  the  CVG  of  CIEN  is 
closest  to  0.90;  for  b  =  16,  both  CIEN  and  CIEQ  yield  the  best  CVG's.   This 
makes  sense  since  VN  (for  b  =  2  and  16)  and  V-  (for  b  =  16)  are  less  biased 
than  V.  and  V_  (Example  1  and  Table  1).  A  related  consequence  is  that  CIEN  and 
CIEft  produce  larger  EHL's  than  those  of  CIE^.   Note  that  the  EHL's  in  Figure  2 
increase  and  then  decrease   as  m  increases  -  the  same  bias-related  pattern  as  for 
the  exact  E[H.]  in  Figure  1.   The  EHL's  for  b  =  16  and  m  £  128  from  Figure  2(d) 
are  more  or  less  in  agreement  with  the  limiting  E[H]/E[HN]  =  1  ratios  in  (14); 
this  is  one  reason  why  we  regard  b  =  16  as  "large." 
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Table  2  has  CVG's  for  the  AR(1)  with  b  =  16,  1  -  a  =  0.90,  and  various 
m  and  <p.   These  roughly  agree  with  the  b  ■+   00  results  from  (10).   We  see  that 
CVG  <  [>]  0.90  when  <p  >  [<]  0.0.  For  any  <p  and  fixed  m,  Table  2  shows  that  the 
CVG's  of  CIEN  and  CIEQ  are  closer  to  1  -  a  than  those  of  CIE.  and  CIE^. 

Results  for  the  EAR(l)  process  with  <p  =  0.9  are  found  in  Figure  3,  whose 
plots  bear  a  striking  resemblance  to  those  of  the  AR(1)  in  Figure  2.   The  only 
notable  difference  between  the  two  figures  is  that,  for  fixed  m,  the  EAR(l)  has 
smaller  CVG's  (see  Table  1).   Since  the  AR(1)  and  EAR(l)  have  the  same 
covariance  structure,  the  EAR(l)'s  poorer  CVG's  are  probably  due  to  its 
exponential  marginals.   It  seems  that  the  effect  of  Bias(V)  on  coverage  is  more 
significant  than  that  of  the  marginals  of  the  X.'s. 

Figure  4  concerns  the  M/M/l  process  with  p  =  0.8.   Again,  the  patterns  in 
the  figure  are  not  much  different  than  those  of  the  AR(1)  and  EAR(l).   The 
M/M/l  simply  requires  more  observations  to  attain  valid  coverage.   The  positive 
serial  correlation  of  the  M/M/l  causes  the  estimators  for  a  to  be  biased  too 
low;  so  poor  coverage  results  for  small  m. 

3.3.2  Additional  Part  I  results 

In  S— K-G,  we  also  give  detailed  discussions  on  (among  other  things): 

-  CIE  performance  for  the  MA(1)  as  6  varies.  We  find  that  the  CIE's 
qualitatively  perform  about  the  same  as  those  for  the  AR(1). 

-  The  sample  -JVar(H)  (SHL)  performance  measure.  For  small  m,  the  SHL's  exhibit 
the  same  general  behavior  as  the  EHL's;  as  m  and  b  become  large,  the  SHL's 
behave  as  in  (14).   Even  though  the  ratios  from  (14)  are  manifested,  the 
differences   between  SHL's  from  competing  methods  are  typically  very  small. 
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-  The  consequences  of  changing  1  -  a.   The  CIE's  qualitatively  perform  the  same 
as  we  vary  1  -  a.   Coverage  is  sensitive  to  decreases  in  1  -  a.  For  example, 
consider  CIE..  for  the  AR(1)  with  <p  =  0.9,  b  =  2,  and  m  =  16;  for  1  -  a  =  0.99, 
0.90,  and  0.80,  Example  5  gives  coverages  of  0.985,  0.852,  and  0.713,  resp. 

3.3.3  Summary  of  Part  I 

For  small  m,  improper  CVG's  were  usually  the  rule.  For  small  m  and  b, 
CIE,.  has  better  CVG's  than  the  other  CIE's,  while  for  small  m  and  large  b,  both 
CIEN  and  CIEn  fared  the  best.  For  fixed  m,  high  CVG  was  most  often  accompanied 
by  high  EHL  and  SHL.  The  CIE's  performed  as  expected  by  asymptotic  theory  when 
m  and/or  b  became  large.  For  large  m  and  small  b,  all  achieved  the  desired 
coverage,  and  the  EHL's  and  SHL's  tended  to  decrease  with  increasing  d.o.f.,  as 
per  Subsection  2.3.   For  large  m  and   b,  the  ratios  from  (14)  took  effect. 

3.4  Results  from  Part  II  of  the  Monte  Carlo  Study 

We  considered  the  MA(1)  (9  =  -0.9),  AR(1)  (<p  =  0.9),  EAR(l)  (<p  =  0.9),  and 
M/M/l  (p  =  0.8),  with  1  -  a  =  0.90  and  d.o.f.  d  =  3  and  15.   Table  3  gives  CVG's 
as  a  function  of  the  sample  size  n.  For  d  =  3,  CIEN,  CIE0,  and  CIEr  perform 
about  the  same  in  terms  of  CVG;  CIE.  fares  poorly  for  small  n.  For  d  =  15  and 
small  n,  CIE..  does  a  bit  better  than  the  others  in  terms  of  CVG;  CIE.  is  not 

U  A 

competitive.   However,  the  performance  of  CIEn  is  "more  variable"  than  the 
other  CIE's  over  the  range  of  stochastic  processes,  d,  and  n  under  study.   For 
instance,  the  CVG  of  CIEn  sometimes  significantly  overshoots  1  -  a,  especially 
for  small  d  (though  this  is  understandable  since  0BM  was  designed  for 
large  b).  As  n  grows  (for  d  =  3  or  15),  it  appears  that  CIE..,  CIE0,  and  CIEp 
achieve  CVG  =  0.90  at  about  the  same  n. 
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4.  DISCUSSION 


We  first  discuss  the  causes  of  improper  CIE  coverage;  why  are  some  CIE's 
better  than  others?  We  then  consider  the  question  of  which  CIE  is  "best?" 

4.1  Causes  of  Improper  Coverage 

A  CIE  of  the  form  in  (1)  will  attain  perfect  coverage  if  its  associated 

pivot  T  =  (I  -  ii)/(V/n)*  ~  t(d);  this  requires  (i)  I     ~  Nor(>i,a2/n) ,  (ii)  V  ~ 

op  =      ** 

a  X  (d)/d,  and  (iii)  independence  of  X  and  V.   Requirement  (i)  is  satisfied  if 

the  marginal  distribution  of  the  X.'s  is  normal.   If  the  X.'s  are  not  symmetric, 

then  T  may  be  skewed  for  small  n.  But  in  most  cases,  a  central  limit  theorem 

asserts  that  (i)  approximately  holds  as  n  grows.  We  believe  that  (ii)  is  the 

key  requirement.   At  a  minimum,  V  must  be  approximately  unbiased  as  an 

2      2 
estimator  of  a  (or  a  ).   In  fact,  since  variance  directly  affects  the  CIE's 

length,  we  claim  that  Bias(V)  is  often  the  main  cause  of  improper  coverage  (at 

least  for  small  m) ;  see  below.   Concerning  (iii),  Glynn  (1982)  and  Kang  and 

Goldsman  (1990)  demonstrate  that  asymmetry   in  coverage  is  directly  related  to 

dependence  between  1     and  V.  However,  Kang  and  Goldsman  give  examples  which 

show  that  actual  coverage  is  not  necessarily  affected  by  such  dependence.   We  do 

not  view  dependence  between  X  and  V  as  a  direct  cause  of  improper  coverage. 

We  first  analyze  the  effect  of  Bias(V)  on  requirement  (ii)  by  examining  V  s 

<j   V/E[V]  instead  of  V;  note  that  E[V]  =  a  .  To  illustrate,  we  shall  use  the 

NOBM  estimator  on  the  AR(1)  and  EAR(l)  processes  with  <p  =  0.9.   For  the  AR(1) 

t%  p   p 

with  b  =  2,  Example  5  says  that  V„  ~  E[V„]X  (1),  and  so  V„  ~  a  X  (1);  for  this 
case,  correction  for  bias  results  in  precisely  the  desired  distribution. 

p  p 

Empirical  p.d.f.'s  of  VN/a  and  V„/a  (based  on  100000  independent  runs)  are 

plotted  in  Figure  5  for  the  AR(l)  and  EAR(l)  with  b  =  8  and  various  m.   For  the 

2  2 

AR(1),  the  sample  p.d.f.'s  of  VN/a  approach  the  X  (7)  p.d.f.  as  m  increases; 
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p  p 

the  corrected  VN/c  is  nearly  (but  not  quite)  X  (7)  (Figures  5(a)  and  5(b)). 
The  empirical  p.d.f.'s  for  the  EAR(l)  exhibit  similar  behavior,  except  that  the 
convergence  to  the  X  (7)  is  slower  (Figures  5(c)  and  5(d)).   Thus,  for  these 
examples,  correction  for  bias  mitigates  the  violation  of  (ii).  Comparing  the 
AR(1)  results  with  the  EAR(l)'s,  we  conclude  that  the  effect  of  Bias(V)  is 
greater  than  that  of  the  marginal  distribution. 

We  will  next  show  for  the  stochastic  processes  investigated  that 
correcting  for  Bias(V)  results  in  good  CIE's.   The  notation  CCVG  in  Table  1  is 
the  exact  (from  Example  5)  or  sample  coverage  obtained  for  the  AR(l)  and  EAR(l) 
models  with  <p  =  0.9  and  b  =  2  and  16.  For  the  AR(1)  with  b  =  2  and  any  m,  the 
corrected  NOBM  pivot  (fR  -  ji)/(VN/n)*  ~  t(l).   So  the  CCVG's  for  CIEN  are 
exactly   1  —  or  =  0.90;  thus,  for  this  example,  Bias(V„)  is  the  sole  cause  of 
improper  coverage.   The  corresponding  MC  CCVG  improvements  for  the  EAR(l)  with 
b  =  2  are  significant  but  not  as  good  as  those  for  the  AR(l),  this  indicating 
that  a  secondary  marginal  distributional  effect  is  present.   This  conclusion  is 
illustrated  yet  again  by  Figure  6.   (The  EAR(l)'s  empirical  p.d.f.'s  are 
somewhat  skewed  for  small  m  as  explained  earlier.)  The  results  from  Table  1 
and  Figures  5  and  6  suggest  that,  for  small  m,  there  are  also  tertiary 
contributors  to  improper  coverage,  perhaps  inter-batch  correlation. 

4.2  Which  is  the  Best  CIE? 

The  question  of  which  CIE  is  the  "best"  depends  on  the  criteria  being 
used.   A  CIE  is  first  judged  by  the  validity  of  its  coverage.   But  the  MC  work 
showed  that  a  CIE  with  good  coverage  might  produce  relatively  wide  half- 
lengths;  so  the  E[H]  and  Var(H)  measures  can  not  be  ignored.   Since  coverage, 
E[H],  and  Var(H)  are  functions  of  the  stochastic  process,  the  CIE  in  use,  the 
number  of  batches  b,  the  batch  size  m,  and  the  level  a,  we  can  see  that  the 
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determination  of  the  "best"  CIE  is  not  straightforward. 

Results  (12)  and  (13)  say  that  E[H]  and  Var(H)  decrease  in  the  d.o.f.  d  as 
m  becomes  large  with  fixed  b;  so  the  more  d.o.f.  the  better  (although  Schmeiser 
1982  finds  that  there  is  little  to  be  gained  by  taking  d  >  30).  Nevertheless, 
it  would  be  incorrect  to  conclude  that  CIEr  (which  has  the  largest  d.o.f.  of 
those  CIE's  under  study)  is  always  the  best.   The  Part  I  MC  results  showed  that 
for  small  b,  CIEN  required  the  smallest  value  of  m  to  achieve  valid  coverage 
(for  large  b,  CIE,.  and  CIE0  required  the  smallest  m);  if  coverage  were  the  only 
criterion  for  CIE  comparison,  CIE„  would  be  declared  the  best  -  not  CIE^.  This 
shows  that  large  sample  superiority  does  not  necessarily  extend  to  the  small 
sample  case.      Indeed,  we  did  not  include  the  STS  "NOBM+maximum"  CIE  from 
Schruben  (1983),  which  has  4b-l  d.o.f.  (and  hence  superior  asymptotic 
properties),  since  it  exhibited  poor  small  sample  performance  compared  to  the 
other  CIE's  (including  CIE.).   For  instance,  for  the  AR(1)  with  <p  =  0.9,  1  -  a  = 
0.90,  b  =  2,  and  m  =  16,  64,  and  256,  the  NOBM+max  CIE  attained  CVG's  (based  on 
1000  independent  simulation  runs)  of  0.397,  0.606,  and  0.776,  resp.;  CIE. 
achieved  CVG's  of  0.684,  0.849,  and  0.895,  resp.  (Table  1). 

Another  basis  for  comparison  among  CIE's  is  to  determine  which  requires 
the  smallest  sample  size  n  to  achieve  valid  coverage  for  some  fixed  d.o.f.  d. 
This  was  the  aim  of  Part  II  of  the  MC  study,  where  CIEN>  CIEQ,  and  CIEp  fared 
more  or  less  the  same;  CIE.  was  not  competitive.   So  there  was  no  clear  winner 

A 

using  the  criterion  of  coverage  under  fixed  d.o.f. 

We  can  still  make  some  recommendations  (for  fixed  sample  size 
procedures).   All  of  the  CIE's  studied  here  are  easy  to  use.   Batch  means  is 
the  simplest  method  to  understand.  All  are  asymptotically  valid  as  the  batch 
size  m  grows;  but  it  "never  hurts  too  much"  to  use  CIEN  (in  comparison  to  the 
other  CIE's)  in  case  m  is  not  "large  enough."  The  price  to  be  paid  when  m  is 
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small  is  that  E[HN]  and  Var(HN)  are  larger  than  their  competitors,  particularly 
when  b  is  also  small.   If  the  user  is  somehow  confident  that  the  batch  sizes 
are  large  enough  to  achieve  valid  coverage,  then  the  user  should  fix  the  d.o.f. 
(perhaps  between  15  and  30  for  the  "usual"  a  values)  and  select  from  among 
CIEN,  CIE0,  and  CIE^.   However,  one  of  the  most  difficult  open  questions  in 
simulation  output  analysis  is  the  determination  of  "sufficient  batch  size"  (cf. 
Fishman  1978  and  Schmeiser  and  Song  1989). 

5.  SUMMARY  AND  CONCLUSIONS 

We  studied  the  behavior  of  different  CIE's  with  special  emphasis  placed  on 
small  sample  size  performance  for  various  stationary  stochastic  processes.  If 
the  achieved  coverages  are  at   the  desired  levels,    and  if  the  total  number  of 
observations  n  is  fixed,  the  ranking  of  the  CIE's  with  respect  to  E[H]  and 
Var(H)  is  determined  by  the  d.o.f.  each  CIE  has.   In  this  case,  the  CIE  with 
the  largest  d.o.f.  has  the  smallest  mean  and  most  stable  confidence  interval 
length. 

Perhaps  our  most  important  finding  was  that,  in  small  sample  settings,  a 
CIE  with  more  d.o.f.  may  not  actually  be  "better"  than  a  competing  CIE;  some 
CIE's  may  require  more  observations  than  others  before  the  asymptotics 
necessary  for  CIE  validity  manifest  themselves.   Quite  often,  the  CIE's  with 

the  highest  d.o.f. 's  performed  the  most  poorly  in  terms  of  coverage! 

2      2 
The  bias  of  V  as  an  estimator  of  a  (or  a  )  plays  a  significant  role  in 

CIE  performance  -  the  less  bias  the  better.  For  instance,  when  m  and  b  are 

fixed,  the  relative  performance  of  the  CIE's  with  respect  to  coverage  is 

directly  related  to  Bias(V).   A  secondary  factor  in  CIE  performance  concerns 

the  underlying  marginal  distribution  of  the  X.'s.  Further,  with  fixed  m  (and 
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even  more  so  with  fixed  n) ,  coverage  often  deteriorates  as  b  or  a  increase; 

this  is  partially  due  to  the  fact  that,  in  these  cases,  t,  .      /0  decreases. 

d,l-oc/<j 

Which  CIE  should  one  use  in  practice?  If  the  sample  size  n  is  "large 

enough,"  we  could  probably  argue  successfully  for  CIEN,  CIEft,  or  CIEp  with 

common  d.o.f.,  15  <  d  <  30.   In  comparison  to  the  other  methods,  CIE,.  is 

probably  the  "safest"  small  sample  method.   There  are  several  interesting 

research  lines.  We  would  like  to  see  more  emphasis  on  small  sample  results, 

including  sequential  procedures  (which  were  not  investigated  here).   Another 

question  concerns  the  fact  that  for  fixed  d.o.f.,  CIEN,  CIE0,  and  CIEp 

achieve  approximately  valid  coverage  for  about  the  same  sample  size. 

Further,  a  good  batch  size  estimation  procedure  would  be  of  tremendous 

import. 
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APPENDIX 


Proof  of  (2)-(5):  We  will  use  the  following  facts: 

l\-i  »'  ■  2irP  -  lli  'p1  -  p[1'(k+1^yi]  •         ca-1) 

11  1  P  1_1  (1-p) 

For  a  covariance  stationary  process  \X.\  with  Y     =  Cov(X.,X.   .), 

a2  =  n-Vard  )  =  Yn  +  -7n"J  (n-i)Y..  (A-2) 

n  v  n7  0       n^i=l   v       7    l  v       7 

For  the  AR(1),  Yfc  =  <plkl,  and  so 

2   l+(p   2<p(l-(pn)     .   2   ..     2   1+$  (k   0x 

a  =  -r—z " — 3— %  and  a  =  lim   a  =  -r—1-  .  (A-3) 

n   1_<p    n(l-<p^2  n"MD  n   1_(p 
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Since 


—  E[VM]   =  yb   .EpC2     ]-bE[X2]  =  b^E[X2]-E[X2]^  =  b[Var(X  )-Var(X  )],  (A-4) 

m      L  NJ       £,j  =  l        j,mJ  L  nJ  *    L  mJ      L  nJ  *  L  m  v  n  J  v       7 


result   (2)   follows  by   (A-3)    and  simplification  (see  Moran  1975) 


// 


We   also  have     (™+l)(™)  E[Vj   =  f  "™+1  E[(X(i,m)  -  X  )2] 

nm  L   0J        ^1=1  l\    v    »    /  n/    j 


=   (n-m+l)[Var(y  +  Var^)]  -  2  ^+1  E[X(i,m)Xn] 


(A-5) 


Since  |i  =  0,     mnE[X(i,m)Xn]  =  j£j  ^  E[Xi+.Xk]  =  j^J"1  Z^  Yj-k  U"6) 

=  r.m+i-1    ryj         j-k       yn  k-j->       ym+i-1  1  +  y  -  yj  -  y""^1 

ij  =  i       lik=l  v  ^k=j+l  v       *       ^j  =  i  1-tp 


=  m 


l+(p         •  <     mw   i         n-m+2  -\\,,A     \i 
.—1     -   (i-y  )(<p     +  <p  ip     )/(l-(p) 


(A-7) 


We  obtain  (3)  by  substituting  (A-3)  and  (A-7)  into  (A-5). 


// 


(4)  follows  from  (A. 5-15)  of  Goldsman  (1984);  (5)  follows  from  (2)  and  (4) 


// 


Proof  of  (6)-(9):  The  MA(l)  has  covariance  function  Y  =  1  +  6  ,  Y   =  9,  and 


Y  =  0,  otherwise.  By  (A-2) , 


c2  =  (1+e)2  -  29/n   and    a2  =  lim   a2  =  (1+e)2. 
n   v  '  '  n-KD  n   v   7 


(A-8) 


Result  (6)  then  follows  from  (A-4)  and  (A-8). 


// 


Note  that 


yn 

M<=1  j-1 


(1+e)      if  1  <  j  <  n 

2 
(1+e)  -e   if  j  =  1  or  n 
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So  by  (A-6), 


f  m(l+e)' 


mnE[X(i,m)X  ]  =  « 


if  1  <  i  <  n-m+1 


>  m(l+e)  -e   if  j  =  1  or  n-m+1 


(A-9) 


We  obtain  (7)  by  substituting  (A-8)  and  (A-9)  into  (A-5) . 


// 


(8)  follows  from  (A. 5-23)  of  Goldsman  (1984);  (9)  follows  from  (6)  and  (8). 


// 


Proof  of  independence  in  Examples  3,  4,  and  5:  Let  X'  =  (Xt,...,X  ),  where  the 

X.'s  are  stationary  normal  with  covariance  matrix  E.  Suppose  G  is  an  nxn 

symmetric  matrix  and  V  =  X'GX.  Problem  1.22  of  Muirhead  (1982)  says  that  V  and 

X  are  independent  iff  l'SG  =0',  where  1'  [0r]  is  a  lxn  vector  of  l's  [0's]. 

Song  and  Schmeiser  (1989)  note  that  VM  =  X'GMX  and  V.  =  X'G.X,  where 
6  /  N   ~  N~     A   ~  A~ 


GN  = 


r  j  -ji 
.-j    jJ 


for  b  =  2,   and  G.  =  (h.h.)  for  b  =  1, 

A     i  J 


where  J  is  an  n/2 x n/2  matrix  consisting  of  l's,  and  h.  =  (n+l)/2  -  i, 


i=l,...,n.  Then  G„  and  G.  both  meet  Muirhead's  condition  for  any  Z.  /, 
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Tabled  -  Some^Results  for  the  AR(l)  and  EAR(l)  Models  with  <p  =  0.9.  All 
entries  for  E[V]  are  exact.  Results  for  CVG  and  CCVG  are2exact(*)  or  are  based 
on  2000  independent  simulation  runs.  For  these  models,  a  =  19. 


E[V] 

AR(1) 

EAR(l) 

(or  <xn) 

b 

=  2 

b  = 

16 

b 

=  2 

b  = 

16 

b=2 

b=16 

CVG 

CCVG 

CVG 

CCVG 

CVG 

CCVG 

CVG 

CCVG 

m  =  4 
NOBM 

0.86 

2.68 

0.744* 

0.900* 

0.479 

0.866 

0.625 

0.798 

0.465 

0.810 

OBM 

0.58 

2.64 

0.453 

0.840 

0.466 

0.859 

0.337 

0.673 

0.446 

0.804 

Area 

0.31 

0.31 

0.418 

0.907 

0.183 

0.888 

0.296 

0.701 

0.164 

0.799 

Comb 

0.49 

1.46 

0.432 

0.878 

0.365 

0.862 

0.321 

0.683 

0.337 

0.806 

(an 

6.19 

16.19) 

m  =  16 

NOBM 

6.10 

9.27 

0.852* 

0.900* 

0.756 

0.896 

0.791 

0.850 

0.742 

0.872 

OBM 

4.22 

9.23 

0.673 

0.864 

0.750 

0.890 

0.609 

0.773 

0.733 

0.866 

Area 

2.85 

2.85 

0.684 

0.896 

0.491 

0.891 

0.615 

0.800 

0.480 

0.866 

Comb 

3.94 

5.96 

0.689 

0.884 

0.651 

0.893 

0.625 

0.795 

0.646 

0.868 

(   2 
(CTn 

13.57 

18.30) 

m  =  64 

NOBM 

14.79 

16.02 

0.891* 

0.900* 

0.874 

0.904 

0.881 

0.889 

0.869 

0.892 

OBM 

12.84 

16.00 

0.841 

0.883 

0.862 

0.894 

0.810 

0.841 

0.867 

0.888 

Area 

11.29 

11.29 

0.849 

0.897 

0.801 

0.894 

0.822 

0.866 

0.803 

0.884 

Comb 

12.45 

13.58 

0.852 

0.895 

0.840 

0.898 

0.819 

0.858 

0.841 

0.888 

(   2 
(an 

17.59 

18.82) 

m  =  256 

NOBM 

17.95 

18.25 

0.898* 

0.900* 

0.899 

0.905 

0.900 

0.902 

0.886 

0.890 

OBM 

17.30 

18.25 

0.895 

0.904 

0.902 

0.905 

0.900 

0.909 

0.890 

0.897 

Area 

16.90 

16.90 

0.895 

0.902 

0.886 

0.903 

0.888 

0.895 

0.878 

0.896 

Comb 

17.25 

17.56 

0.890 

0.901 

0.897 

0.908 

0.896 

0.902 

0.880 

0.892 

(   2 
(CTn 

18.65 

18.96) 

m  =  1024 

NOBM 

18.74 

18.81 

0.900* 

0.900* 

0.897 

0.899 

0.883 

0.883 

0.909 

0.910 

OBM 

18.56 

18.81 

0.905 

0.907 

0.897 

0.898 

0.915 

0.918 

0.902 

0.903 

Area 

18.47 

18.47 

0.901 

0.905 

0.893 

0.897 

0.893 

0.895 

0.898 

0.901 

Comb 

18.56 

18.64 

0.892 

0.896 

0.900 

0.902 

0.895 

0.897 

0.899 

0.903 

(   2 

18.91 

18.99) 
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Table  2  -   CVG  Results  for  the  AR(1)  Process,  b  =  16,  1  -  a  =  0.90. 
All  entries  are  based  on  at  least  1000  independent  simulation  runs. 


(p     -0.9    -0.5     0.0     0.5     0.9 


m  =  4 

NOBM 

0.962 

0.945 

0.913 

0.823 

0.479 

0BM 

0.966 

0.949 

0.908 

0.816 

0.466 

Area 

1.000 

0.974 

0.898 

0.677 

0.183 

Comb 

0.998 

0.971 

0.900 

0.755 

0.365 

m  =  16 

NOBM 

0.951 

0.915 

0.905 

0.887 

0.756 

0BM 

0.958 

0.922 

0.906 

0.895 

0.750 

Area 

0.988 

0.942 

0.908 

0.860 

0.491 

Comb 

0.981 

0.926 

0.910 

0.880 

0.651 

m  =  64 

NOBM 

0.918 

0.904 

0.903 

0.901 

0.874 

0BM 

0.929 

0.916 

0.910 

0.908 

0.862 

Area 

0.943 

0.901 

0.885 

0.885 

0.801 

Comb 

0.931 

0.900 

0.896 

0.894 

0.840 

m  =  256 

NOBM 

0.915 

0.903 

0.904 

0.900 

0.899 

0BM 

0.915 

0.911 

0.909 

0.909 

0.902 

Area 

0.931 

0.914 

0.918 

0.917 

0.886 

Comb 

0.921 

0.918 

0.916 

0.915 

0.897 

m  =  1024 

NOBM 

0.906 

0.901 

0.899 

0.899 

0.897 

0BM 

0.903 

0.899 

0.897 

0.897 

0.897 

Area 

0.907 

0.904 

0.906 

0.904 

0.893 

Comb 

0.909 

0.906 

0.902 

0.900 

0.900 
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Table  3  -  CVG  Results  for  Variance  Estimators  with  Common  d.o.f. 


All  entries  are 

based 

on  2000 

independent  simulation  runs 

• 

n  = 

32 

64 

128 

256 

512 

1024 

2048 

4096 

8192 

ARCl). 
NOBM 

to  =  0.9 
0.688 

.  d  =  3 
0.787 

I 
0.848 

0.880 

0.889 

0.899 

0.896 

0.902 

0.905 

OBM 

0.692 

0.792 

0.834 

0.874 

0.896 

0.910 

0.903 

0.923 

0.920 

Area 

0.505 

0.674 

0.795 

0.858 

0.880 

0.900 

0.895 

0.889 

0.912 

Comb 

0.701 

0.799 

0.847 

0.886 

0.883 

0.903 

0.895 

0.905 

0.908 

EAR  CD 
NOBM 

.  to  =   0. 
0.622 

9.  d  = 
0.736 

3 
0.826 

0.862 

0.887 

0.900 

0.896 

0.903 

0.896 

OBM 

0.631 

0.725 

0.820 

0.864 

0.887 

0.906 

0.917 

0.912 

0.916 

Area 

0.426 

0.630 

0.771 

0.844 

0.868 

0.886 

0.893 

0.893 

0.893 

Comb 

0.636 

0.740 

0.822 

0.863 

0.881 

0.898 

0.894 

0.902 

0.896 

M/M/l, 
NOBM 

o  =  0.8 
0.437 

.  d  =  3 
0.541 

0.632 

0.707 

0.768 

0.804 

0.836 

0.862 

0.877 

OBM 

0.432 

0.540 

0.627 

0.704 

0.752 

0.799 

0.842 

0.867 

0.886 

Area 

0.290 

0.420 

0.533 

0.624 

0.708 

0.769 

0.817 

0.835 

0.855 

Comb 

0.442 

0.548 

0.634 

0.701 

0.758 

0.806 

0.838 

0.860 

0.871 

MA(l), 
NOBM 

e  =  -0. 
0.985 

9.  d  = 
0.981 

3 
0.986 

0.975 

0.953 

0.932 

0.931 

0.920 

0.911 

OBM 

1.000 

1.000 

1.000 

1.000 

0.998 

0.986 

0.971 

0.959 

0.944 

Area 

0.994 

0.993 

0.989 

0.980 

0.967 

0.955 

0.947 

0.933 

0.918 

Comb 

0.994 

0.992 

0.989 

0.976 

0.963 

0.932 

0.933 

0.919 

0.908 

AR(1). 
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0.381 

,  d  =  15 
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0.893 

0.897 

0.910 

Area 

0.126 

0.189 

0.318 

0.513 

0.695 

0.809 

0.871 

0.891 

0.903 

Comb 

0.385 

0.491 

0.649 

0.755 

0.834 

0.876 

0.894 

0.903 

0.907 

EAR(l) 
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,  to  =  0. 
0.324 

9.  d  = 
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0.897 

0.897 

OBM 

0.319 

0.505 

0.660 
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0.877 

0.899 

0.891 
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Area 

0.076 

0.173 

0.303 

0.496 

0.670 

0.802 
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0.479 

0.615 
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